Transmission of media information such as audio signal, video signals, still images etc. is typically based on packetization i.e. information to be transmitted is framed into packets. The packets are then transmitted as one or more packet streams. The packet streams can be transmitted e.g. as RTP (Real Time Protocol) packets or as packets of an other protocol which are not necessarily RTP packets. It may happen that some of the packets get lost for some reason during the transmission. For example, the transmission channel may be affected by disturbances which may weaken the signals which carry the packet stream and cause losses to the packet stream. In the transmission stage it is possible to add some error recovery information to the packet stream which can be used at the receiving stage to recover lost packets. One known method is based on forward error correction (FEC) in which extra packets carrying error recovery information are inserted to the packet stream. Such extra packets are called as repair RTP packets in this description i.e. packets carrying error recovery information. The repair RTP packets are formed on the basis of those packets which should be protected by the repair RTP packets. For example, a bitwise XOR operation is performed on data of the packets and the calculated FEC values are packetized to form the repair RTP packets. The repair RTP packets and the packets which are used in forming the repair FEC packets forms an FEC block. Therefore, the packet losses of an FEC block may be recoverable by using the information of the repair RTP packets of the same FEC block.
The same media information may be delivered to multiple receivers by using e.g. the Multimedia Broadcast/Multicast Service (MBMS) streaming delivery method. In this method there is not a separate media stream for each receiver but different receivers receive the same media stream. Some of the packets need to be stored (buffered) in the receiver before the decoding and playback of the media stream can be started. There is also a need to buffer packets of the media stream during the reception of the media stream.
When the streaming delivery method of MBMS is in use, media buffering needs to be controlled to minimize delay and allocated memory. The decoding of the media stream of RTP packets requires initial buffering in the receiving device because if the receiver started to decode source RTP packets (i.e. media RTP packets) immediately when the first one is received, any lost source RTP packet would cause a delay in decoding until the repair RTP packets of the FEC block are received. This would consequently cause a pause in the playback.
The duration to receive an FEC block, i.e. the time which elapses from the reception of the first bit of the first RTP packet of the FEC block until the reception of the last bit of the last RTP packet of the same FEC block (as well as interleaving of source and repair RTP packets), varies, because the size of source RTP packets varies in terms of bytes and the number of source RTP packets and the number of repair RTP packets in an FEC block varies.
It may happen that the first FEC block that the receiver receives is shorter in terms of bytes compared to some of the succeeding FEC blocks. Consequently, initial buffering of one FEC block entirely is not a sufficient condition to guarantee pauseless decoding and playback. Therefore, senders should give receivers information that allows sufficient but not exhaustive amount of initial buffering.
As the MBMS is a multicast/broadcast service, new receivers may “tune in” in the middle of the stream i.e. new receivers may begin to receive the stream later than the first packet of the stream was transmitted. The optimal (minimum) initial buffering delay is usually not constant throughout the stream.
Size of FEC Decoding Buffer
As was shown above, received RTP packets have to be buffered before the decoding of the FEC packets can be started. Such a buffer is called as an FEC decoding buffer in this description. The buffer occupancy level of the FEC decoding buffer depends inter alia on 1) the transmission schedule of source and repair RTP packets, 2) the amount of initial buffering before starting the emptying of the buffer, 3) the method of building an FEC matrix inside the FEC decoding buffer, and 4) the output rate of data from the FEC decoding buffer. The maximum buffer occupancy level determines the required buffer size for the stream. It is evident that in different receiving device implementations the FEC decoding and the related buffering may be implemented differently compared to each other. For example, players may have a different approach for output rate handling—one device may push data out from the FEC decoding buffer as soon as the buffers “downstream” (i.e. decoder input buffers) allow, and another device may pull data out from the FEC decoding buffer just on time when the next piece of data is needed for decoding. Therefore, the maximum buffer occupancy level may vary in different implementations, and consequently it is problematic to determine the required FEC decoding buffer size of a particular stream.
Encoders and transmitters should also be aware of the supported FEC decoding buffer size of all receivers when performing FEC encoding and transmission scheduling.